Extrinsic Jensen-Shannon Divergence: Applications to Variable-Length Coding

Abstract—This paper considers the problem of variable-length coding over a discrete memoryless channel (DMC) with noiseless feedback. The paper provides a stochastic control view of the problem whose solution is analyzed via a newly proposed symmetrized divergence, termed extrinsic Jensen–Shannon (EJS) divergence. It is shown that strictly positive lower bounds on EJS divergence provide non-asymptotic upper bounds on the expected code length. Strictly positive lower bound on EJS divergence, and hence non-asymptotic upper bounds on the expected code length, are obtained for the following two sequential coding schemes: posterior matching and MaxEJS coding scheme which is based on a greedy maximization of the EJS divergence. As an asymptotic corollary of the main results, this paper also provides a rate–reliability test. Variable-length coding schemes that satisfy the condition(s) of the test, are guaranteed to achieve the capacity (and the optimal error exponent). The results are specialized for posterior matching and MaxEJS to obtain a deterministic one-phase coding scheme achieving the capacity and the optimal reliability. For the special case of symmetric binaryinput channels, simpler deterministic schemes are proposed and analyzed.